Directed Replacement

This paper introduces to the nite-state calculus a family of directed replace operators. In contrast to the simple replace expression, UPPER -> LOWER, de ned in Karttunen (1995), the new directed version, UPPER @-> LOWER, yields an unambiguous transducer if the lower language consists of a single string. It transduces the input string from left to right, making only the longest possible replacement at each point. A new type of replacement expression, UPPER @-> PREFIX ... SUFFIX, yields a transducer that inserts text around strings that are instances of UPPER. The symbol ... denotes the matching part of the input which itself remains unchanged. PREFIX and SUFFIX are regular expressions describing the insertions. Expressions of the type UPPER @-> PREFIX ... SUFFIXmay be used to compose a deterministic parser for a \local grammar" in the sense of Gross (1989). Other useful applications of directed replacement include tokenization and ltering of text streams.